(1) Field of the Invention
The present invention relates to airborne imaging software and, more particularly, to a software system and method for simultaneous display of multiple geo-tagged videos acquired at a particular geographical location at different times. The invention is especially useful for dual-or-multiple displays of videos collected on an oil or gas pipeline Right Of Way (ROW) at different times to monitor encroaching threats to pipeline integrity.
(2) Description of Prior Art
Federal regulation requires that pipeline operators monitor their right of ways in an effort to prevent accidental damage to the pipeline and to detect leaks. As a result, oil and natural gas pipeline companies regularly monitor hundreds of thousands of miles of pipeline in the continental United States, and more on a multi-national scale. This is no small task. Conventional monitoring entails flying pilot/observers in light aircraft over the pipeline. The routine occurs usually at two week intervals. The aircraft flies at low altitude (300 ft to 500 ft) and the pilot, observer or both visually inspect the right of way.
The current industry practice is to have the pilot/observer inspect and document observations on paper. There would be great benefit if technology could be used to collect aerial data and: a) create a record of the condition of the ROW for process improvement purposes; and b) develop techniques for automatically detecting threats to pipeline integrity such as encroaching machinery. An emerging trend is to have the aircraft collect aerial imagery data for later viewing. This often involves stabilized aerial imaging using collection systems that correlate the imagery with geographic information. Geographic Information System (GIS) video integrates spatial location information with the video to help the viewer interpret the information. For example, U.S. Pat. No. 5,633,946 to Lachinski et al. (Geospan Corporation) issued May 27, 1997 shows an early method and apparatus for collecting and processing visual and spatial position information from a moving vehicle. This patent teaches how to record for each frame of the video both the time code and current spatial position information provided by a GPS receiver and inertial data from an IMU. In this way, each recorded video image can be correlated with the spatial position of the camera at the time the image was recorded.
After the flight, sitting at a computer, an analyst can replay the video, observe any issues, and identify the location of those issues. However, the analyst remains at a disadvantage if asked to compare two georeferenced videos taken at different times to determine if the observed issue represents a change in the condition of the ROW. Currently, this requires the analyst to manually load the current video and an archived video and to manually maintain their geographic progression. This is complicated by the fact that an aircraft flown on different days experiences different wind speeds and directions. Consequently, it will always follow slightly different paths through the air, will always fly at different speeds, and will always experience different roll, pitch and yaw conditions as the aircraft progresses down the ROW. A more efficiently established side-by-side comparison is important to effectively monitor threats to pipeline integrity, machinery encroachments over time, or other temporal trends of interest such as vegetation patterns, drought monitoring, downed trees, deforestation, fire hazard monitoring, and soil erosion patterns. This is acknowledged by Slaymaker, Using Georeferenced Large-Scale Aerial Videography As A Surrogate For Ground Validation Data, Winrock—Remote sensing for forest environments, Chapter 8, page 474 (2003), who suggests a visual comparison between wide-angle video images and a false color-infrared of each satellite image to correctly identify the location of individual trees in the corresponding video.
There are a number of prior art references for comparing images in real-time as they are observed. For example, U.S. Pat. No. 5,045,937 to Myrick (Space Island Products) issued Sep. 3, 1991 shows a method for geographical surveying using multiple cameras to obtain split-screen images with overlaid geographical coordinates.
U.S. Pat. No. 6,766,226 to Andersen (Andersen Aeronautical Technologies) issued Jul. 20, 2004 shows a method of monitoring utility lines with aircraft using a dual sensor camera capable of concurrently capturing both thermal images and real time video images. The thermal images may be simultaneously reviewed with the video images for visual anomalies.
The foregoing and other known prior art synchronize videos as they are taken by applying a sync signal to the cameras, which is then used in playback to produce a split screen effect. However, video that has adequate resolution to identify threats requires a large amount of computer memory. For example, data collected on a recent flight test suggests that the video file size can exceed 5 gigabytes per mile. When considering that several hundred thousand miles are flown every two weeks, the data storage and review process rapidly compounds, and synchronization is useless for this.
Geotagging is the well-known process of adding geospatial metadata to digital media including still photographs and video, among others. The geospatial metadata usually includes latitude and longitude coordinates. There are several known formats used for recording these coordinates, the most common being degrees, minutes, and seconds (DMS). However, DMS format, coordinates can alternately be reported in “decimal degrees” (DD) that presents the minutes and seconds as a fraction of one degree.
It would be greatly advantageous to provide a master-slave video playback interface in which a master video player is time based, and a slave video player displays a second video georeferenced to the first, e.g., displaying the georeferenced frames of the second video in closest proximity to those being shown on the master player. This would allow more efficient simultaneous viewing of multiple geo-tagged videos acquired at a particular geographical location to compare videos collected at different times. This would be especially useful for dual-or-multiple displays of videos collected on an oil or gas pipeline Right Of Way (ROW) at different times to monitor encroaching threats to pipeline integrity.